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f^i ' Abstract 

(N ! 

In this paper, it is shown that the rank function of a matroid can be represented by a "mutual 
information function" if and only if the matroid is binary. The mutual information function 
considered is the one measuring the amount of information between the inputs (binary uniform) 
and the output of a multiple access channel (MAC). Moreover, it is shown that a MAC whose 
mutual information function is integer valued is "equivalent" to a linear deterministic MAC, in 
the sense that it essentially contains at the output no more information than some linear forms of 
the inputs. These notes put emphasis on the connection between mutual information functionals 
and rank functions in matroid theory, without assuming prior knowledge on these two subjects. 
The first section introduces mutual information functionals, the second section introduces basic 
notions of matroid theory, and the third section connects these two subjects. It is also shown that 
entropic matroids studied in the literature correspond to specific cases of MAC matroids. 

^ ! 1 Information Measures 
in 

t^. . Definition 1. Let X and y be two finite sets called respectively the input and output alphabets 
and let M(X) denote the set of probability measures on X. A channel W with input alphabet X 
CN ' and output alphabet y is a collection of conditional probability measures {VF(-|x) € M(y) : x € X}. 
For fixed alphabets, we denote the set of channels by M{y\X). 

Definition 2. The mutual information of a probability measure /x € M{X x y) is defined by 
i>< : Ifa) = D(ji\\nx * »y) = E„log £ , 

& . 

where [ix and \iy are respectively the marginals in X and y of [i. 

If X and Y are two random variables on respectively X and y, then I(X; Y) denotes where 
fj, is the joint distribution of X, Y. 

If P G M (X) is an input distribution and W € M(y\X) is a channel, then I(P, W) denotes I(fi) 
where fj, = P o W. 

The uniform mutual information (UMI) of a channel W E M{y\X) is given by I(W) := I(Ux°W), 
where Ux is the uniform distribution on X. 

For a given channel W and for any input distribution Px, I(Px, W) has the following operational 
meaning in information theory: it is an achievable rate for reliable communication on a discrete 
memoryless channel with transition probability W. In particular, I(W) is an achievable rate and the 
largest achievable rate is given by the capacity C = maxp^MtX) W)- 

Definition 3. A multiple access channel (MAC) W with m users, input alphabet X and output 
alphabet y, is a channel having input alphabet X m and output alphabet y, i.e., an element of 
M(y\X m ). A binary MAC is a MAC for which X = F 2 . 
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Let E m = {1, . . . ,m}. 

Definition 4. The mutual information function (MIF) of a MAC W G M(y\X m ) with input distri- 
butions Pi, ... , P m € M(X) is defined by the function 

I(P x ,...,P m ,W) : 2 Em — >■ R 

5 ^/[5](Pi,...,P m ,W) :=/(X[S] ; y,A[S c ]), (1) 

where 

(I[B m ],F)~(P 1 x...xP m )oI 

If Pi = ... = P m = we call this function the uniform mutual information function (UMIF) and 
we denote it by I(W) (the same notation is used for the single-user mutual information, which is not 
a conflicting notation since single-user channels correspond to 1-user MACs). 

The operational meaning of the MIF is the following: the region 

{(Pi, . . . , R m ) : < Ri ^ I(X[S];YX[S C ]), S C E rn } 

represent achievable rates on a memoryless MAC W, when the m users are not allowed to cooperate 
during the communication. (If the m users were allowed to cooperate, rates given by I(P, W) for 
any P £ M(X m ) would be achievable.) If there are no restriction on the input distributions, the 
closure of the convex hull of all such regions (for any input distributions) gives the capacity region. 

2 Matroids 

Definition 5. A matroid M is an ordered pair (E,X), where E is a finite set called the ground set 
and X is a collection of a subsets of E called the independent sets, which satisfies: 

(71) € X. 

(12) HI El and I' C I, then I' £ 1. 

(13) If iijijj € 1 and < I/2I) then there exists an element e € I2 — I\ 
such that I\ U {e} G X. 

We then say that M is a matroid on E with independent sets X. 
Definition 6. Let M be a matroid given by (E,X). 

• A basis is a maximal (with respect to the inclusion) subset of E which is independent. The 
collection of bases is denoted by B. Note that all the subsets of the bases are the independent 
sets. Hence, a matroid can be defined by its bases. 

• An dependent set is a subset of E which is not independent. The collection of dependent sets 
is denoted by T> = X c . 

• A circuit is a minimal (w.r. to the inclusion) subset of E which is dependent. The collection 
of circuits is denoted by C. 

Definition 7. On any matroid M, we define a rank function r : V(E) — > Z + such that for any 
S C E, r(S) is given by the cardinality of a maximal independent set contained in (or equal to) S. 
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Note: one should check that this is a well defined function, i.e., that any two maximal independent 
sets in S have the same cardinality. This is actually due to the fact that all the bases in a matroid 
have the same cardinality. This also implies that r(E) is given by the cardinality of a basis. We 
denote R := r(E). 

Lemma 1. The rank function satisfies the following properties. 



Note: all the objects that we have defined so far (independent sets, dependent sets, bases, circuits, 
rank function) can be used to define a matroid, i.e., we can define a matroid as a ground set E with 
a collection of circuits or a ground set E with a rank function, etc. Moreover, each of these objects 
can be characterized by a set of axioms, as for example in the following lemma. 

Lemma 2. Let E be a finite set and r : V(E) -4 Z+. We have that r is a rank function of a matroid 
on E if and only if r satisfies (Rl), (R2) and (R3). 

Definition 8. A vector matroid over a field F is a matroid whose ground set is given by the column 
index set of a matrix A defined over F, and whose independent sets are given by the column index 
subsets indicating linearly independent columns. We denote such a matroid by M = M [A]. We call 
A a representative matrix of the matroid. 

For a vector matroid, the objects defined previously (dependent sets, bases, rank function) nat- 
urally match with the objects defined by the corresponding linear algebraic definition. The matroid 
theory is also connected to other fields such as graph theory. For an undirected graph, the set of edges 
define a ground set and a collection of edges that does not contain a cycle defines an independent 
set. A major problem in matroid theory, consist in identifying whether a given matroid belongs to a 
certain class of structured matroids, such as vector matroids or graphic matroids. We are particularly 
interested here in the problem of determining whether a given matroid can be expressed as a vector 
matroid over a finite field. 

Definition 9. A matroid is representable over a field F if it is isomorphic to a vector matroid over 
the field F. A ¥ 2 representable matroid is called a binary matroid. 

Note that there are several equivalent representation matrices of a given representable matroid 
over a field. It easy to show that on a rank R matroid which is representable on F, one can always 
pick a representative matrix of the form [J#|A], where A is an R x (n — R) matrix. This is called a 
standard representative matrix. 

We review here some basic construction defined on matroids. 

Definition 10. Let M be a matroid with ground set E and independent sets X. Let S C E. 

• The restriction of M to S, denoted by M\S, is the matroid whose ground set is S and whose 
independent sets are the independent sets in I which are contained in (or equal to) S. 

• The contraction of M by S, denoted by M/S, is the matroid whose ground set is E — S and 
whose independent sets are the subsets I of E — S for which there exists a basis B of MIS' such 
that IUB EX. We will see an equivalent definition of the contraction operation when defining 
the dual of matroid. 



(Rl) 
(R2) 
(R3) 



IfXCE, then r(X) < \X\. 

IfX 1 CI 2 C£, then r(JTi) < r(X 2 ). 

IfXi,X 2 C E, then 

r(Xi U X 2 ) + r{X 1 n X 2 ) < r{X{) + r(X 2 ). 
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• A matroid N that is obtained from M by a sequence of restrictions and contractions is called 
a minor of M. 



We now define a matroid which is particular with respect to the representability theory of binary 
matroids. 

Definition 11. Let m, n € Z + with m < n. Let E be a finite set with n elements and B the 
collection of m-element subsets of E. One can easily check that this collection determines the bases 
of a matroid on E. We denote this matroid by U mn and call it the uniform matroid of rank m on a 
set with n elements. 

The following are two major theorems concerning the representation theory of binary matroids. 

Theorem 1. [TutteJ A matroid is binary if and only if it has no minor that is U 2 ^. 

Theorem 2. [Whitney] A matroid is binary if and only if the symmetric sum (A) of any two circuits 
is the union of disjoint circuits. 

Remark 1. In a binary matroid, the circuit space of a matroid is equal to the kernel of its representa- 
tive matrix. Indeed, if we multiply a circuit C by the representative matrix A, we are summing the 
columns corresponding to a circuit. But this sum must be 0, since a circuit is a minimal dependent 
set, and therefore, we can express one of the columns as the sum of the others. 

Next, we introduce the duality subject, which will play a central role in the applications of our 
next section. 

Theorem 3. Let M be a matroid on E with a set of bases B. Let B* = {E — B : B G B}. Then B* 
is the set of bases of a matroid on E. We denote this matroid by M* and call it the dual of M . 

Lemma 3. If r is the rank function of M , then the rank function of M* is given by 



We can then define the contraction operation via duality. 
Definition 12. The contraction of M by S is given by the dual of the restriction of M* on S, i.e., 



We conclude this section with the definition of polymatroids. 

Definition 13. A polymatroid is an ordered pair of a finite set E called the ground set and a /3-rank 
function p : V(E) — > M + which satisfies 



r*(S) 



r(S c ) + \S\-r(E). 



M/S = {M*\S)*. 



(Rl) 
(R2) 
(R3) 



/(0) = 0. 

If X x C X 2 C E, then f(X t ) < f(X 2 ). 
If Xx,X 2 C E, then 

f{x 1 u x 2 ) + f(x x n x 2 ) < f(x x ) + f(x 2 ). 



The region of M m defined by {(Ri, ■ ■ ■ , R m ) : Rs < f(S), S C E} is called a polyhedron. 



We refer to [7j for more details on matroid theory. 
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3 Extremal Dependencies 



This section connects the two previous ones, by characterizing MACs having an integer valued UMIF, 
i.e., a matroidal UMIF. Note that there exists a wide class of problems connecting information and 
matroid theory, such as characterizing entropic matroids; we refer to [2 HI [5l EJ (U [10] and references 
therein, and we show in Section 14.21 that entropic matroids are particular cases of MAC matroids. 
An application of the results presented here is given in [I], for a MAC polar code construction. 
Recall that E m = {1, . . . , m}. 

Theorem 4 (g]). For any m>l, any MACW € M{y\X m ) and any Pi, ... , P m € M(X) the func- 
tion p = I (Pi, . . . , P m , W) defined in ([ID is a f3-rank function on E m and (E m , p) is a polymatroid. 

We denote this polymatroid by M[P\, . . . , P m , W). We use M[W] when Pi = . . . = P m = Ux- 
If for a polymatroid M we have M = M[W] (where = means isomorphic), we say that W is a 
representative channel of M. 

In this section, we are interested in characterizing the MACs for which the function p is integer 
valued, i.e., for which (E m , p) defines a matroid. We restrict ourselves to binary MACs and we only 
consider the case where P\, . . . ,P m are all given by the uniform distribution. One can easily come 
up with examples of binary MACs that would provide an integral p. But we are mostly interested 
in the reverse problem, i.e., in characterizing the matroids that admit such a mutual information 
representation. From a communication point of view, such MACs are interesting because they are 
trivial to communicate over with respect to both noise and interference management, and they indeed 
correspond to the extremal MACs created in the polarization process of PQ. 

Definition 14. A matroid M is a BUMAC matroid if M = M[W] for a binary MAC W. Hence, a 
BUMAC matroid is a matroid whose rank function is given by the UMIF (Definition 0]) of a binary 
MAC. "BUMAC" refers to binary uniform MAC. 

Theorem 5. A matroid is BUMAC if and only if it is binary. 

To prove this theorem, we first prove the following lemma. 

Lemma 4. J7 2 ,4 is not BUMAC. 

Proof. Assume that the rank function of t/2,4 is the UNIF of a MAC. We then have 



for all i,j,k,l distinct in {1,2,3,4}. Let yo be in the support of Y. For x £ F|, define P(x\yo) = 
W(yo\x)/ YlzeF% W(yo\z). Then from ([3]), P(0, 0, *, *|y ) = for any choice of *, * which is not 
0,0 and P(0, 1, *, *|yo) = f° r an Y choice of *,* which is not 1,1. On the other hand, from (J2|), 
P(0, 1, 1, 1 12/0) must be equal to po. However, we have form ([3]) that P(l, 0, *, *|yo) = for any choice 
of *, * (even for 1, 1 since we now have P(0, 1, 1, l\yo) > 0). At the same time, this implies that the 
average of P(l, 0, *, *\yo) over *,* is zero. This brings a contradiction, since from ([2]), this average 
must equal to po. □ 

Proof of Theorem [5l We start with the converse. Let M be a binary matroid on E with representative 
matrix A. Let D be the deterministic channel defined by the matrix A, then we clearly have 



I(X[i,j};Y) = Q, 
I(X[i,j];YX[k,l})=2, 



(2) 
(3) 



M ^ M[D]. 
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For the direct part, let M be a BUMAC matroid. We already know from Lemma 2] that M 
cannot contain U2.4 as a minor. If instead C/2,4 is obtained by a contraction of S c from M , i.e., 
M/S c = U 2A , it means that (M*\S)* = U 2A . Since U 2A is self dual, we have M*\S = f7 2 ,4- Let us 
denote by r* the rank function of M* . We have for any Q C i£ 

r *(Q) = |Q| +r (Q c )- r ( J E), 

= |Q| + I(X[Q C ];YX[Q]) 
= |Q|-/(X[Q] ; y), 

where the last equality follows from the chain rule of the mutual information. Since ?**(•) restricted 
to 5 is the rank function of U 2 ^, we have in particular 

r*(T) = 2, VTcS s.t. |T| = 2 
r*(S) = 2, 

that is, 

2-/(X[T] ; y) = 2, VT C 5 s.t. \T\ = 2 (4) 
A-I(X[S];Y) = 2. 

This implies, by the chain rule of the mutual information, 

I(X[T];YX[S -T]) = 2, VT C S s.t. |T| = 2. (5) 

Hence, from the proof of Lemma HI (J4j) , (EJ) cannot simultaneously hold and C/2,4 cannot be a 
minor of M. From Tutte's Theorem (cf. Theorem [1]), M is binary. □ 

Previous theorem gives a characterization of BUMAC matroids. Note that, if we were interested 
in characterizing binary matroids through BUMAC matroids, then the following corollary holds. 

Definition 15. A BULMAC matroid is a BUMAC matroid with linear deterministic representative 
channel. 

Corollary 1. The family of binary matroids is isomorphic to the family of BULMAC matroids. 

We now formally establish the connection between extremal MACs and linear deterministic 
MACs. 

Theorem 6. Let W be a binary MAC with m users whose UMIF is integer valued, i.e., M\W\ is a 
binary matroid. Let A be a matrix representation of M\W\ and let Y be the output of W when the 
input X[E m ] (with i.i.d. uniform components) is sent. Then 

L(AX[E m )-Y) = rankA = I{X[E m ];Y). 

This theorem says that for a binary MAC with integer valued UMIF, the output of i.i.d. uniform 
inputs contains all the information about the corresponding linear form of the inputs and nothing 
more. In that sense, MACs with integer valued UMIF are "equivalent" to linear deterministic MACs. 
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Proof. Let M = M[W] with M[W] = M[A] and let us assume that M has rank R. Let B be the set 
of bases of M and let B* be the set of bases of M*. Since r(B) = \B\ = R for any B G B, we have 

r{B) = I{X[B];YX[B C ]) = R, VBe B. (6) 

Moreover, the rank function of M* is given by 

r*(S) = \S\-I(X[S];Y) 

and for all D G £>*, we have r*(D) = \D\ = \E m \ — R. Hence 

r*(D) = \E m \ -R=\E m \-R- I(X[D];Y), VD G B* , 

or equivalently 

7(X[Z)];y) =0, VDeB*. (7) 

Hence, form ([6]) and (|7|) and the fact that B* = {E m — B : B G £>}, we have 

I(X[B];YX[B C }) = r, VBeB, (8) 
I(X[B C ];Y) = 0, V5GB. (9) 

Note that ([8]) means that if any realization of the output Y is given together with any realization 
of X[B C ], we can determine Moreover, @ means that X[£? c ] is independent of Y. Let us 

analyze how these conditions translate in terms of probability distributions. Let yo G Supp(Y). We 
define 

Po(x) := W(y \x)/ ]T W(y \x), Vx G Fg\ 

From ([8]), if po(x) > 0, we must have po(x') = for any x such that a/ [B c ] = x[B c ] for some B c G 
From Q, we have that 

Po(x) = 2 R ~ m , MB G B, x[B c ] G ¥™~ R . 

x :x [B c ]=x[B c ] 

Hence, for any B G 23 and any x[B c ] G ¥™~ R , we have 

x' :x' [B c ]=x[B c ] 
x :x [B c ]=x[B c ] 

Let * := 2 R ~ m . Previous constraints imply that po(x) G {0, *} for any x G F™ and that the number of 
x with po(x) = * is exactly 2l- Em l~ r . Let us assume w.l.o.g. that po(0) = *, where is the all vector. 
Note that we know one solution that satisfies previous conditions. Namely, the solution that assigns 
a * to all vectors belonging to KerA As expected, dimKer(A) = \E m \ — rank(^4) = \E m \ — r. We 
want to show that there cannot be any other assignment of the *'s in agreement with the matroid M. 
In the following, we consider elements of F™ as binary vectors or as subsets of E m , since F™ — 2 Em . 
The field operations on F™ translate into set operations on 2 Em , in particular, the component wise 



iR—m 



-iR—m 



(10) 

(11) 
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modulo 2 addition x\ + x<i of binary vectors corresponds to the symmetric different x\t\X2 of sets, 
and the component wise multiplication x\ -X2 of binary vectors corresponds to the intersection x\ f]X2 
of sets. 

We now check which are the assignments which would not violate (jlOp and (llip . We have assumed 
w.l.o.g that is assigned *, hence is assigned *. From (|10p . any x for which x[B c ] = for some 
B € £>, must be assigned 0. Note that 

x[B c ] = = x-B c = = x<ZB = xel, 

where I is the collection of independent sets of M. Hence, the elements which are assigned by 
checking the condition (jlOf) are the independent sets of M, besides which is assigned *. 
For B € B and s £ F™, we define 

1(B) := {I G 1 : I C B} 

and 

T S (B) := {x : x[B c ] = s[B c }}. 
Note that I S {B) = s+l(B), indeed: 

x[B c ] = s[B c ] = x-B c = s-B c = (x + s)-B c = = x + sCB = xe s+X(B). 

Now, if r(S) = r(T) for two sets S and T with T Q S, we have 

Jpf[S-T];YX[S c ]) =0. 

This means that (Y, X[S C ]) is independent of X[S — T]. From the point of view of probability 
distributions, this means that 

w x[ s-T]\Yxis4 x i s - T }\yo x i sc V = 2Ffr> yx i s ~ T ^ x i sc ] 

or equivalently, 

Hence, if we set the components of a; € F™ to frozen values on S°, then, no matter how we freeze 
the components of x on T, the average of po(-) on S — T must be the same. 

Let C 6 C be a circuit. By the definition of circuits, if we remove any element of C we have a 
basis of M\C. Let -B be such a basis, we then have r(C) = r(B). We now want to freeze the values 
on C c and B in two ways. 

1. If we pick d = CDB c , then 

I d (B) = {x : x CC - B}. 

These are the elements that are strictly contained in C, i.e., elements of I, including 0. Therefore, 
the average of po(') must be ★ for this freezing. 

2. If we pick d = C, we already know that the average of po(-) must be *, but we have 

I d (B) = {x : x + C CC - B}. 
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These are the elements containing B, possibly elements of C — B but nothing else. Therefore, the 
options are x = Corx€Z — 0. This forces C to be assigned *. 

Hence, we have shown that all circuits of M must be assigned *. This in turns imply several 
other assignments. Namely, 

JC + (X-0) (12) 

cec 

must be assigned 0. 

Let us next consider a union of two disjoint circuits, D = C\ U C 2 . Then, if we remove any single 
elements of D, say by removing an element of C\, we obtain a disjoint union of an independent set 
and a circuit, say IU C<i. Hence, 

r{C 1 UC 2 ) = r{IUC 2 ). 

We can then use the same technique as previously, but this time, we need to use that (I12|) is 
assigned 0. Note that is important to assume that the union is disjoint, in order to guarantee that 
C 2 + IUC 2 = I £l. 

We can then use an induction to show that any union of disjoint circuits must be assigned *. 
Finally, for a binary matroid, any symmetric difference of two circuits is given by a union of disjoint 
circuits (this can be directly checked but notice that it is contained as one of the implications of 
Theorem [2] due to Whitney). Hence, the space generated by the circuits, seen as elements of (F™, +) 
must be assigned *, and using Remark [H we conclude the proof since we have assigned the 1/* 
numbers of * without any degrees of freedom, and the assignment is done on KerA □ 

3.1 Recursions using mutual information properties 

In this section we re-derive some of the results of previous section using inductive arguments. We 
start by checking a result similar to Theorem [6] for the case m = 3. 

Lemma 5. Let W be a binary MAC with 2 users. Let X[E 2 ] with i.i.d. uniform binary com- 
ponents and let Y be the output of W when X[E] is sent. If I(X[1];YX[2]), I(X[2];YX[1]) and 
I(X[1)X[2];Y) have specified integer values, then I(X[1];Y), I(X[2];Y) and I(X[1] + X[2]; Y) have 
specified values in {0, 1}, and vice-versa. 

Proof. Let 

/ := [I(X[1];YX[2]),I(X[2];YX[1]),I(X[1}X[2];Y)} 
J := [I(X[l];Y), I(X[2];Y),I(X[1] + X[2];Y)}. 

Note that by the polymatroid property of the mutual information, we have 

/ € {[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1], [1, 1, 2]}. (13) 

Let y E supp(Y) and for any x E F| define ¥{x\y) = W(y\x)/ ^ 2 eF 2 ^(^l 2 ) (recall that W is the 
MAC with inputs X[1],X[2] and output Y). Assume w.l.o.g. that p := P(0,0|y) > 0. 

• If / = [0, 0, 0] we clearly must have J = [0, 0, 0]. 
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• If / = [*, 1, 1], we have I(X[2]; = 1 and we can determine X[2] by observing X[l] and 
Y, which implies 

P(01|y) = 0. 

Moreover, since I(X[1];Y) = I(X[1]X[2];Y) — I(X[2];YX[1]) = 0, i.e., X[l] is independent of 
Y, we must have that ^x[2] ^( x [l]- x [2]|y) is uniform, and hence, 

P(00|y) = 1/2, P(10|y) + P(ll|y) = 1/2. 

Now, if -k = 1, by a symmetric argument as before, we must have P(ll|y) = 1/2 and hence we 
the input pairs 00 and 11 have each probability half (a similar situation occurs when assuming 
that F(x\y) > for x ^ (0, 0)), and we can only recover X[l] + X[2] from Y, i.e., J = [0, 0, 1]. 
If instead * = 0, we then have I(X[2];Y) = I(X[1]X[2];Y) - I(X[1];YX[2]) = 1 and from a 
realization of Y we can determine X[2], i.e., P(10) = 1/2 and J = [0, 1,0]. 

• If I = [1, 0, 1], by symmetry with the previous case, we have J = [1, 0, 0]. 

• If / = [1, 1, 2], we can recover all inputs from Y, hence J = [1, 1, 1]. 

For the converse statement, Note that J must be given by [0, 0, 0], [0, 1, 0], [1, 0, 0], [0, 0, 1] or 
[1, 1, 1]. Clearly, the case [0, 0, 0] implies I = [0, 0, 0]. 

For the case J = [0,1,0], note that I(X[2];Y) = 1 implies h(X[2]\Y) = 0, i.e., for any 
y € supp(y), /i(X[2]|y = y) = 0. This means that for any y £ supp(Y), if p2(^[2]jy) > for 
some x[2], we must have p2(x[2]\y) = for x[2] ^ x[2]. We use pi, i = 1,2, for the probability dis- 
tribution of X[i] given the realization Y = y and p\2 for the probability distribution of (X[1],X[2]) 
given Y = y. Assume now (w.l.o.g.) that Pi2(0, 0|y) > 0. Since p2(a^[2]|y) = X^ril Pi2(^[l]^[2]|y), 
previous observation implies that pi2(01|y) = pi2(ll|y) = 0. Moreover, 7(X[l];y) = implies that 
/i(X[l]|y = y) = 1, i.e., for any realization of Y, the marginal of X[l] is uniform, which implies 
Pi2(00|y) = Pi2(10|y) = 1/2. Hence, if we are given the realization of X[l] and Y, we can decide 
what X[2] must be, and this holds no matter which values of (X[1],X[2]) is assigned a positive 
probability, i.e., /(J[2];7X[1]) = 1. If instead we are given X[2] and Y, we can not infer anything 
about X[l], i.e., I(X[1];YX[2}) = 0. Finally, by the chain rule, I(X[1]X[2];Y) = 1. The case where 
[I(X[1];Y),I(X[2];Y),I(X[1] + X[2]\Y)] is equal to [1,0,0] can be treated symmetrically and the 
other cases in a similar fashion. □ 

Lemma 6. Let W be a binary MAC with m users. Let X[E m ] with i.i.d. uniform binary components 
and let Y be the output ofW when X[E] is sent. If I(X[S];YX[S C ]) has a specified integer value for 
any S C E m , then I(X[E m ] ■ S;Y) has a specified value in {0,1} for any S C E m , and vice-versa. 
Note: X[E m ] ■ S = ® ie sX[i\ 

The recursive argument for the proof of the direct part of this Lemma has been proposed by 
Eren §a§oglu [8] and contains the idea behind this section. The direct statement in the Lemma is a 
consequence of Theorem [6] but is proved here using the recursive approach. 

Proof. Let I[S}(W) be assigned an integer for any S C E m . By the chain rule of the mutual 
information 

I(X[E m ];Y) = I(X[S];Y)+I(X[S C ];YX[S]), 
and we can determine /(XfS 1 ]; Y) for any S. Since for any T C S 

I(X[S];Y)=I(X[T];Y)+I(X[S-T];YX[T]), 
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we can also determine I(X[S];Y X[T]) for any S,T C with S D T = 0. Hence we can determine 

I(X[1],X[2];YX[S}) 

I(X[1]-YX[S]X[2]) 

I(X[2];YX[S]X[1\) 

and using Lemma El we can determine 

I(X[1]+X[2];YX[S}) 

for any S C ,E m with {1, 2} ^ 5", hence 
for any i,j £ £; m . 

Assume now that we have determined I(Y2t X[i]', ^AfS"]) for any T with |T| < /c and S" C E m — T. 
Let T = {1, . . . , /c} and let 5 C {A; + 2, ... , m}. 

lC£x[{\,X[k + l];YX[S]) 

T 

= I(X[k + 1};YX[S]) + lC£m,YX[S}X[k + 1]), 

T 

in particular, we can determine 

I(X[k + l];Y^2x[i},X[S]) 

T 

= I(J2x\i],X[k + l];YX[S}) 

T 
T 

and 

I(J2x\i],X[k + l];YX[S}) 

T 

lC£m,YX[S]X[k + l}) 

T 

I(X[k + l];Yj2x[i\,X[S}) 

T 

and using Lemma we can determine 

I{Y,X[i\+X[k + l\-YX[g\) 

T 

hence 



ii 



for any T C E m with \T\ = k + 1. Hence, inducting this argument, we can determine 1 (^2 T X[i];Y) 
for any T C .E^. 

For the converse statement, assume that we are given I(X[E m ] ■ S;Y) € {0, 1} for any S C E m . 
In particular, I(Xi; Y), I(Xi] Y) and I(Xi + Xj; Y) is determined for any i, j € E m , and hence, from 
Lemma we have that I(Xi;YXi) and I{XiXj\Y) are determined (and integer valued) for any 
i, j G E m . 

Note that we can also determine I(X[E m ] ■ T;YX[i\) for any T C E m and i € -E m — T; indeed, 
we know I(X[E m ] ■ T; Y) for any T C so for i £ 2? m — T, we know 

/(X[i]+X[£ TO ]-T;Y), (14) 
I(X\i};Y), (15) 

/(x[E m ]-r ; y), (16) 

and hence, using Lemma [5l we can determine I(X[E m ] ■ T;YX[i]). 

Let us assume now that we have determined I(X[S];YX[F — S]) for any F such that \F\ < k 
and S C F, as well as J(X[^ m ] • T;yi[K]) for any K such that < k - 1 and T C F m - K. 
We have already checked that this can be determined for k = 2. We now check that we can also 
determine these quantities for k + 1 instead of k. 

Let K with \K\ = k — 1. Assume w.l.o.g. that 1,2,3 ^ K. Since we assume to know 

I(X[1];YX[K]), (17) 
I(X[1]+X[2];YX[K}), (18) 
I(X[1] + X[2]+X[3];YX[K]), (19) 

using Lemma [H we can determine J(X[1] + X[2];VX[X U 3]). Using a similar argument we can 
determine I(X[E m ] ■ T;YX[K]) for any K such that \K\ < k and T C £' m — K. Moreover, since we 
now know I(X[1] + X[2];YX[K]) and also 

I(X[1];YX[K}), (20) 

I(X[2];YX[K}), (21) 

we can determine with Lemma [5] 

I(X[1];YX[KU2}), (22) 

I(X[2]-YX[KU1}), (23) 

/(I[l]I[2];FI[if]), (24) 

and hence, we can determine I (X[K\\;Y X[K2\) for |fTi| < 2 and \K±\ + |i^2| < fc + 1. From the 
chain rule of the mutual information, we have 

I(X[1]X[2]X[3];YX[K - 3]) = I(X[1]X[2];YX[K - 3]) + I(X[3];YX[K - 3]X[l]X[2]) (25) 

and both term in the right hand side above are already determined. Hence, by iterating the chain 
rule argument, we can determine I(X[S]; YX[F — S]) for any F such that \F\ < k + 1 and SCF. 
Finally, we can iterate these arguments on k to reach F = E m , i.e., to determine an integer for 
I(X[S];YX[S C }) for any S C E m . 

□ 
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3.2 Quasi-Extremal Channels 

In this section, we provide technical steps necessary to extend previous results to polymatroids which 
are "close" to matroids. 

Lemma 7. Let W be a binary MAC with m users. Let X[E m ] with i.i.d. uniform binary components 
and let Y be the output ofW when X[E] is sent. Let e > 0, if I(X[S];YX[S C ]) has a specified value 
in Z + (—£, e) for any S C E m , then I(X[E m ] ■ S; Y) has a specified value in [0, o £ (l)) U (1 — o £ (l), 1] 
for any S C E m . Note: X[E m ] ■ S = © ieS X[f] 

The converse of this statement also holds. This lemma follows from the results of previous sections 
and from the following lemmas. 

Lemma 8. For two random variables X, Y such that X is binary uniform and I(X; Y) < e, we have 

Pr{y : \\P x \ Y (-\y) - < e l/2 } > 1 - 21n2 £ 1 / 2 , 

where U is the binary uniform measure. 
Proof. Since I(X;Y) < e, we have 

D{P xy \\P x Py) <e 

and from Pinsker's inequality 

)P-Q\\i<D(P\\Q) 



21n2' 
we get 

\\PxY - 

y 

Therefore, by Markov's inequality, we have 

Pr{y:||P X | y (.|y)-C/(-)||i>a}<^£ 

a 

and by choosing a = e 1 / 2 , we get the desired inequality. □ 

Lemma 9. For two random variables X,Y such that X is binary uniform and h(X\Y) < e, define 
E £ by 

y £E e Pr{X = 0\Y = y} Pr{X = 1\Y = y} < e, 

then 

Pr{E £ } >l-7(e), 

with 7(e) — >• when e — > 0. 

This lemma tells us that if Pr{X = 0|Y = y} is not small, we must have that Pr{X = \\Y = y} 
is small with high probability. It is given as a problem in [3]. 
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4 Extensions 



4.1 g-ary Matroids 

The results of last sections are expected to generalize to the q-ary alphabet case, where q is a prime 
or power of prime. In particular, we have the following. 

Claim: A matroid is g-ary representable if and only if its rank function is given by the UMIF of 
a MAC with q-ary inputs. 

Hence, one could equivalently characterize q-ary matroids by characterizing rank functions which 
are representable by q-ary alphabets MAC. 

4.2 Entropic matroids 

The following result can be found in [2]. 

Lemma 10. Let Z[E m ] be an m- dimensional random vector with arbitrary distribution over F™. 
Then the function r : S i— > H{Z[S}) is a {3-rank function and (E m ,r) is a polymatroid. 

Hence, we can define a notion of entropic matroid, which is a matroid whose rank function is 
representable by an entropic function as above. 

We now show that entropic matroids can be studied as specific cases of MAC matroids. Consider 
a specific MAC which consist of an additive noise perturbation of the input, i.e., 

Y[E m ] = X[E m ]@Z[E m ], 

where Xf.E'm] is an m-dimensional random vector with i.i.d. uniform components over ¥ q and Z[E m ] 
is an m-dimensional random vector of arbitrary distribution on F™, independent of X [E m ]. Then, 

I(X[S];YX[S C }) = \S\ - H(X[S]\YX[S C }) 

= \S\ - H(Y[S] Q Z[S]\Y, (Y[S C ] Q Z[S C })) 

= \S\ - H(Z[S]\Y,Z[S C }) 

= \S\-H(Z[S]\Z[S C ]) 

= H(Z[S]) + \S\-H(Z[E m }). 

Hence, an entropic matroid corresponds to a particular case of MAC matroid which has additive 
noise. 

References 

[1] E. Abbe and E. Telatar, Polar codes for the m-user multiple access channel and matroids, 
International Zurich Seminar on Communications (IZS), Zurich, March 2010. Available on 
larXiv:1002.0777 vl [cs.IT]. 

[2] S. Fujishije, Polymatroidal dependence structure of a set of random variables, Information and 
Control, vol. 39, pp. 55-72, 1978. 

[3] R. Gallager, Information Theory and Reliable Communication, John Wiley & Sons, 1968. 

[4] T. S. Han, A uniqueness of shannon's information distance and related nonnegativity problems, 
J. Comb., Inform. Syst. Sci., vol. 6, no. 4, pp. 320-331, 1981. 



14 



[5] L. Lovasz, Submodular functions and convexity, in Mathematical Programming - The State of the 
Art, A. Bachem, M. Grdtschel, and B. Korte, Eds. Berlin: Springer- Verlag, 1982, pp. 234-257. 

[6] F. Matus, Probabilistic conditional independence structures and matroid theory: background, Int. 
J. of General Systems 22, pp. 185-196. 

[7] J. Oxley, Matroid Theory, Oxford Science Publications, New York, 1992. 

[8] E. §asoglu, private communications. 

[9] D. Tse and S. Hanly, Multi-access Fading Channels: Part I: Polymatroid Structure, Optimal 
Resource Allocation and Throughput Capacities, IEEE Trans. Inform. Theory, vol. IT-44, no. 7, 
pp. 2796-2815, November 1998. 

[10] Z. Zhang and R. Yeung, On characterization of entropy function via information inequalities, 
IEEE Trans, on Information Theory, vol. 44, no. 4, pp. 1140-1452, 1998. 



15 



